6.1 Dimensionality Reduction

Previously in the course, we have discussed algorithms suited for a large number of data points. This lecture discusses when the dimensionality of the data points becomes large. We denote the data set as x1, x2, . . . , xn ∈ RD for D >> n, and will consider dimensionality reductions f : RD → Rd for d << D. We would like the function f to preserve some properties of the original data set, such as variance, correlation, distances, angles, or “clusters”. For a concrete example, consider consider linear functions,